Search for: All records

The ingenious approach of Chatterjee (2021) to estimate a measure of dependence first proposed by Dette et al. (2013) based on simple rank statistics has quickly caught attention. This measure of dependence has the appealing property of being between 0 and 1, and being 0 or 1 if and only if the corresponding pair of random variables is independent or one is a measurable function of the other almost surely. However, more recent studies (Cao & Bickel 2020; Shi et al. 2022b) showed that independence tests based on Chatterjee’s rank correlation are unfortunately rate inefficient against various local alternatives and they call for variants. We answer this call by proposing an improvement to Chatterjee’s rank correlation that still consistently estimates the same dependence measure, but provably achieves near-parametric efficiency in testing against Gaussian rotation alternatives. This is possible by incorporating many right nearest neighbours in constructing the correlation coefficients. We thus overcome the ‘ only one disadvantage’ of Chatterjee’s rank correlation (Chatterjee, 2021, § 7).

 

Summary Chatterjee (2021) introduced a simple new rank correlation coefficient that has attracted much attention recently. The coefficient has the unusual appeal that it not only estimates a population quantity first proposed by Dette et al. (2013) that is zero if and only if the underlying pair of random variables is independent, but also is asymptotically normal under independence. This paper compares Chatterjee’s new correlation coefficient with three established rank correlations that also facilitate consistent tests of independence, namely Hoeffding’s $D$, Blum–Kiefer–Rosenblatt’s $R$, and Bergsma–Dassios–Yanagimoto’s $\tau^*$. We compare the computational efficiency of these rank correlation coefficients in light of recent advances, and investigate their power against local rotation and mixture alternatives. Our main results show that Chatterjee’s coefficient is unfortunately rate-suboptimal compared to $D$, $R$ and $\tau^*$. The situation is more subtle for a related earlier estimator of Dette et al. (2013). These results favour $D$, $R$ and $\tau^*$ over Chatterjee’s new correlation coefficient for the purpose of testing independence.